Mbp on T0: Mixing Oating-and Xed-point Formats in Bp Learning

We examine the eecient implementation of back prop type algorithms on T0 4], a vector processor with a xed point engine, designed for neural network simulation. A matrix formulation of back prop, Matrix Back Prop 1], has been shown to be very eecient on some RISCs 2]. Using Matrix Back Prop, we achieve an asymptotically optimal performance on T0 (about 0.8 GOPS) for both forward and backward phases, which is not possible with the standard on-line method. Since high eeciency is futile if convergence is poor (due to the use of xed point arithmetic), we use a mixture of xed and oating point operations. The key observation is that the precision of xed point is suucient for good convergence, if the range is appropriately chosen. Though the most expensive computations are implemented in xed point, we achieve a rate of convergence that is comparable to the oating point version. The time taken for conversion between xed and oating point is also shown to be reasonable.